You can use Python and R scripts as data sources, by downloading or writing a script an converting it into a table. To connect to a scripting source, add the R or Python node from the Sources menu (not from the Scripting menu) and construct or configure the script from its Properties panel.
Scripting can also be used for data manipulation during the ETL. This functionality is configured by added the required scripting node from the Scripting menu.
- Click here to learn more about Scripting for data manipulation purposes.
Scripted Sources Supported
The following scripting languages are supported as data sources:
Write the Script
There are a few ways in which you can provide Pyramid with your script:
Pyramid Marketplace: you can download scripts from the Pyramid Marketplace. Pyramid will then import the dataset and convert it to a table.
Pick a Script: select a script that's been created in Pyramid and saved to the Pyramid content manager.
Write a Script: write or paste a script directly to the scripting window.
Scripting Environments
Pyramid enables Administrators to create multiple scripting environments, so as to allow for multiple language and package versions. This means that you'll need to choose which scripting environment to connect to.
When choosing the environment, you'll be able to view a list of each package (and its version) currently installed on the selected environment.
Output and Data Frame Support
After providing the script, you'll need to define the output, including the data frame, column name, and data type for each column. Pyramid automatically enters the outputs for scripts downloaded from the Marketplace. For other scripts, you can configure the output manually column by column, or you can use the Auto Detect data frame support function. Auto Detect lets you provide the data frame once, and then Pyramid detects the data frame and columns in the script and adds them to the output.
- Click here to learn more about data frame support.
